The present invention relates to data processing by digital computer, and more particularly to testing versions of computer programs.
Applications are used by computer systems, for example, to process (e.g., sort, filter, store) data. To ensure proper processing of the data, the applications are generally tested. In one approach to testing, known inputs are supplied to the application during execution to establish what the predicted proper outputs would be. Using these predicted outputs, another instance of the application running on a possibly different platform and running at any location may be tested by applying the same inputs to the other application and comparing the processed outputs to the predicted outputs. The testing is done by a process that contains testing instructions that deliver a large number of tests to the application and process the results to determine whether features of the application are working properly.
When features of an application are changed or enhanced to produce an updated application or a newer application version, the testing process must be updated, too. In addition, because older versions of the application may continue to be used, the original test process must be maintained and used even though a new test process has been generated.
In general, in one aspect, this invention provides methods and apparatus, including computer program products, implementing and using techniques for testing a computer program application. Information identifying a target system is obtained. Application information identifying the actual version of the application as installed on the target system is obtained. A test script is selected from multiple test script versions based on the actual version of the application and the selected test script is used to test the application as installed on the target system.
Advantageous implementations can include one or more of the following features. The application information can be obtained from meta-data installed on the target system. The meta-data can be generated by a build process building the application. The meta-data can be installed on the target system by an install process installing the application on the target system. The application information can be obtained from the application on the target system. Multiple test script versions can have a common associated parameter interface defining import and export parameters available for each of the multiple test script versions. Each of the multiple test script versions can have associated with it information specifying with which application versions the test script version is suitable for use. A set of input parameter values for a function call in the test script can be obtained, it can be determined whether the set of input parameter values provides a complete set of input parameters for the function call, and the function call can be executed with the set of input parameter values. The function call can be executed only if the set of input parameter values does provide a complete set of input parameters, and otherwise the test script is terminated with an error. The test script can have an associated parameter interface defining input parameters available for use in the test script, and parameter values can be accessed according to the parameter interface to obtain the set of input parameter values for the function call in the test script.
A set of input parameter values can be obtained for a function call in the test script, it can be determined whether the set of input parameter values provides a complete set of input parameters for the function call, default parameter values can be obtained for input parameters for the function call if the set of input parameter values does not provide a complete set of input parameters, and the function call can be executed with the set of input parameter values and the default parameter values. The function call can be executed only if the set of input parameter values and the default parameter values provide a complete set of input parameters and the test script can otherwise be terminated with an error otherwise. The test script can have an associated parameter interface defining input parameters available for use in the test script, and the parameter interface can be used to obtain import and export parameters for the test script.
In general, in another aspect, this invention provides methods and apparatus, including computer program products, implementing and using techniques for handling parameters in a test script. A test script for testing a function of a computer program is executed. The test script includes a command invoking the function and arguments, and the command arguments in the test script include a set of test input parameters for the function. The required input parameters for the function are determined. An error is detected if there is a mismatch between the required input parameters and the test input parameters.
Advantageous implementations can include one or more of the following features. The required input parameters for the function can be determined by obtaining meta-data describing the function as it exists in a particular installation of the computer program. The test script can be terminated and the error can be reported if the test input parameters include more than the required input parameters. The test script can be terminated and the error can be reported if the test input parameters include more than the required input parameters and a strict mode of operation is in effect. The test script can be terminated and the error can be reported if the test input parameters do not include all the required input parameters and no default values have been established for the parameters missing from the test input parameters.
In general, in another aspect, this invention provides methods and apparatus, including computer program products, implementing and using techniques for handling parameters in a test script. A set of test records defining variant parameter values for test parameters is received. A first and a second test script for testing a function of a computer program is received. The test scripts have distinct first and second sets of import parameters, respectively. The first and second test scripts are executed against first and second versions of the computer program, respectively, using the same set of test records to provide import parameter values.
In general, in another aspect, this invention provides methods and apparatus, including computer program products, implementing and using techniques for testing a computer program. A test configuration is received. The test configuration has an associated test script and one or more associated data containers. Each data container defines a set of parameters and one or more sets of variants, each set of variants specify a value for each of the parameters in the set of parameters, and each data container is reusable with multiple test configurations. The test script has a set of test parameters and the test configuration in addition has a set of test parameter variants. Each of the test parameter variants is a set of values for respective test parameters in the set of test parameters. The test parameter variants include one or more references for obtaining data values from each one or more of the associated data containers. The test script is executed to test a computer program. The test script is executed for each of the test parameter variants in the set of test parameter variants.
The invention can be implemented to realize one or more of the following advantages. By using command interfaces to access test data for application testing, repetitive listing of the test data and parameters is reduced in a test script and test scripts condense into manageable sizes. Also, command interfaces may be reused in the test script to improve the efficiency of creating and editing test scripts. Over testing periods, trends may emerge which repetitively use similar command interfaces that may be accessed from a library and not recreated for testing future application functions.
Moreover, command interfaces do not have to be revised when a new version of a test script is written. Superfluous parameters in the command interfaces may be disregarded or omitted from the feature call, and additional parameters that need to be passed to the feature under test may be assigned default values. Where a test script does not use command interfaces, the same principles can be used to adjust feature calls, so that the original version of the test script can be used to test a new version of the feature. In other words, a new version of a feature does not necessarily require a new version of the test script used to test the feature.
In addition, using a table, database, or resource to determine which version of a test script should be used to test a particular version of an application allows for test scripts and test cases to be developed and organized in a coherent fashion from a central location.
The details of one or more implementations of the invention are set forth in the accompanying drawings and the description below. Other features and advantages of the invention will become apparent from the description, the drawings, and the claims.